Priming Mindfulness Project

Comparison & analysis report

Introduction

This report describes the results of the first priming mindfulness study, as well as of the two online pilot studies from the Varela project. It was initially made using Dominique Makowski’s Supplementary Materials template.

This is an exploratory (not confirmatory) study. The result of this exploration will be used to conduct a second, confirmatory, preregistered study.

Packages & Data

Packages

library(rempsyc)
library(dplyr)
library(interactions)
library(performance)
library(see)
library(patchwork)
library(ggplot2)
library(rstatix)
library(DescTools)
library(report)
library(bestNormalize)

summary(report(sessionInfo()))

The analysis was done using the R Statistical language (v4.2.0; R Core Team, 2022) on Windows 10 x64, using the packages bestNormalize (v1.8.2), DescTools (v0.99.44), ggplot2 (v3.3.5), interactions (v1.1.5), performance (v0.9.0), see (v0.7.0), report (v0.5.1), dplyr (v1.0.8), patchwork (v1.1.1), rempsyc (v0.0.4.0) and rstatix (v0.7.0).

Data

data <- read.csv("data/fulldataset.csv")

# Dummy-code group variable
data <- data %>% 
  mutate(condition_dum = ifelse(condition == "Mindfulness", 1, 0),
         condition = as.factor(condition))

cat(report_participants(data))

272 participants ()

report(data$country)

x: 8 entries, such as USA (93.75%); Canada (1.47%); India (1.47%) and 5 others (4 missing)

# Allocation ratio
report(data$condition)

x: 2 levels, namely Control (n = 139, 51.10%) and Mindfulness (n = 133, 48.90%)

Data Preparation

In this stage, we define a list of our relevant variables and standardize them according to the Median Absolute Deviation (MAD), which is more robust to extreme observations than standardization around the mean.

Preparation

# Make list of DVs
col.list <- c("blastintensity", "blastduration", "blastintensity.duration",
              "blastintensity.first", "blastduration.first", 
              "blastintensity.duration.first", "KIMS", "BSCS", "BAQ", 
              "SOPT", "IAT")

# Create new variable blastintensity.duration
data$blastintensity.duration <- (data$blastintensity * data$blastduration)
data$blastintensity.duration.first <- (data$blastintensity.first *
                                         data$blastduration.first)
# Divide by 2? Do some other sort of transformation given I multiplied two scores?
# Should I multiply them after standardization or before?

Blast Intensity * Duration

Why combine the intensity and duration scores? Should we? For a discussion, see:

Elson, M., Mohseni, M. R., Breuer, J., Scharkow, M., & Quandt, T. (2014). Press CRTT to measure aggressive behavior: the unstandardized use of the competitive reaction time task in aggression research. Psychological assessment, 26(2), 419. https://doi.org/10.1037/a0035569

• Bushman and Baumeister (1998) used the sum of volume and duration settings in the first of 25 trials [p. 3]
• Lindsay and Anderson (2000) multiplied volume with log-transformed duration settings. The average over 25 trials of those products was their measure for overall aggression.
• Carnagey and Anderson (2005) averaged the products of volume and the square root of duration to form a single “aggressive energy score” (p. 887). No reason is given for this other than the claim that this single score supposedly is a valid measure and that duration should be square rooted.
• Bartholow, Sestir, and Davis (2005) multiplied the average volume and duration settings to form a composite aggressive behavior score. Although Bartholow, Bushman, and Sestir (2006) also used volume and duration settings, they standardized and summed the two parameters instead of multiplying them.
• Sometimes the option of setting the volume and/or duration to zero as a way to act nonaggressively is provided. Including settings of zero as an option also raises further questions, for example, how to handle trials in which participants set only one of the two intensity parameters to zero. [Note: we do have zero as option]
• With regard to the analysis, there is no definitive answer to the question of how to calculate aggression scores, or whether different scores might measure different types of aggression, as long as none of them have been properly validated. As it seems that volume and duration do not measure the exact same construct, it is advisable to consider them as separate measures for related subdimensions of aggression.

First sound blast

Why use the first sound blast only instead of the average of all trials? Should we?

According to some, the Taylor Aggression Paradigm is not a measure of aggression per say, but of reactive aggression, because participants react to the other “participant’s” aggression. They suggest that for a pure measure of aggression, it is recommended to use only the first sound blast used by the participant before he receives one himself. At this stage, we attempt the analyses with all these different measures of aggression for exploratory purposes. See earlier reference to Elson et al. (2014):

• If researchers are interested in measuring unprovoked aggression, they should also look at the settings in the first trial. Those studying provoked aggression or retaliation, on the other hand, should focus on all trials except the first one.

Preliminary exclusions

First, we know that we only want to keep participants who had at least an 80% success rate in the critical experimental manipulation task. Let’s see how many participants have less than an 80% success rate.

data %>% 
    filter(manipsuccessleft < .80) %>% 
    count()

n
17

There’s 17 people, let’s exclude them.

data <- data %>% 
    filter(manipsuccessleft >= .80)
cat(report_participants(data))

255 participants ()

t-tests

In this section, we will: (a) test assumptions of normality, (b) transform variables violating assumptions, (c) test assumptions of homoscedasticity, (d) identify and winsorize outliers, and (e) conduct the t-tests.

Normality

lapply(col.list, function(x) 
  nice_normality(data, 
                 variable = x, 
                 title = x,
                 group = "condition",
                 shapiro = TRUE,
                 histogram = TRUE))

## [[1]]

## 
## [[2]]

## 
## [[3]]

## 
## [[4]]

## 
## [[5]]

## 
## [[6]]

## 
## [[7]]

## 
## [[8]]

## 
## [[9]]

## 
## [[10]]

## 
## [[11]]

Several variables are clearly skewed. Let’s apply transformations. But first, let’s deal with the working memory task, SOPT (Self-Ordered Pointing Task). It is clearly problematic.

SOPT Exclusions

Let’s do an histogram proper to see if it helps diagnosing the problem with SOPT.

hist(data$SOPT)

That looks weird, there’s some obvious outliers here; they probably didn’t do the task correctly, especially since there’s a gap after 60 errors. Let’s see how many people made more than 60 errors.

data %>% 
  filter(SOPT < -60) %>% 
  count

n
10

There’s 10 people with more than 60 errors. Let’s exclude them.

data <- data %>% 
  filter(SOPT >= -60)
cat(report_participants(data))

245 participants ()

Transformation

Normally, the SOPT raw scores represent the number of errors, but I had multiplied it by -1 initially so that a smaller score would mean lower working memory capacity. Here we reverse it again to be able to use the various transformations. We also add a constant of 1 to avoid scores of zero which can interfere with the transformation (and so the Box Cox transformation can be considered).

data <- data %>% 
  mutate(SOPT = SOPT * -1 + 1)

The function below transforms variables according to the best possible transformation (via the bestNormalize package), and also standardizes the variables.

predict_bestNormalize <- function(var, print.transform = TRUE, standardize = TRUE) {
  x <- bestNormalize(var, standardize = standardize)
  if (print.transform == TRUE) {
    print(cur_column())
    print(x$chosen_transform)
    cat("\n")
  }
  predict(x)
}

set.seed(100)
data <- data %>% 
  mutate(across(all_of(col.list), 
                predict_bestNormalize,
                .names = "{.col}.BN"))

## [1] "blastintensity"
## orderNorm Transformation with 245 nonmissing obs and ties
##  - 142 unique values 
##  - Original quantiles:
##    0%   25%   50%   75%  100% 
##  0.00  3.04  4.76  6.04 10.00 
## 
## [1] "blastduration"
## orderNorm Transformation with 245 nonmissing obs and ties
##  - 201 unique values 
##  - Original quantiles:
##     0%    25%    50%    75%   100% 
##    0.0  792.8 1100.4 1299.6 2000.0 
## 
## [1] "blastintensity.duration"
## Standardized sqrt(x + a) Transformation with 245 nonmissing obs.:
##  Relevant statistics:
##  - a = 0 
##  - mean (before standardization) = 68.97142 
##  - sd (before standardization) = 32.98318 
## 
## [1] "blastintensity.first"
## orderNorm Transformation with 245 nonmissing obs and ties
##  - 11 unique values 
##  - Original quantiles:
##   0%  25%  50%  75% 100% 
##    0    3    7    9   10 
## 
## [1] "blastduration.first"
## center_scale(x) Transformation with 245 nonmissing obs.
##  Estimated statistics:
##  - mean (before standardization) = 1149.265 
##  - sd (before standardization) = 632.9021 
## 
## [1] "blastintensity.duration.first"
## orderNorm Transformation with 245 nonmissing obs and ties
##  - 59 unique values 
##  - Original quantiles:
##    0%   25%   50%   75%  100% 
##     0  2000  7000 13300 20000 
## 
## [1] "KIMS"
## Standardized asinh(x) Transformation with 245 nonmissing obs.:
##  Relevant statistics:
##  - mean (before standardization) = 1.935386 
##  - sd (before standardization) = 0.1339016 
## 
## [1] "BSCS"
## Standardized asinh(x) Transformation with 245 nonmissing obs.:
##  Relevant statistics:
##  - mean (before standardization) = 1.9655 
##  - sd (before standardization) = 0.2242601 
## 
## [1] "BAQ"
## Standardized asinh(x) Transformation with 245 nonmissing obs.:
##  Relevant statistics:
##  - mean (before standardization) = 1.799905 
##  - sd (before standardization) = 0.2964333 
## 
## [1] "SOPT"
## Standardized asinh(x) Transformation with 245 nonmissing obs.:
##  Relevant statistics:
##  - mean (before standardization) = 3.176059 
##  - sd (before standardization) = 0.6035038 
## 
## [1] "IAT"
## center_scale(x) Transformation with 245 nonmissing obs.
##  Estimated statistics:
##  - mean (before standardization) = -0.5083574 
##  - sd (before standardization) = 0.3708143

col.list <- paste0(col.list, ".BN")

Let’s check if normality was corrected.

# Group normality
lapply(col.list, function(x) 
  nice_normality(data, 
                 x, 
                 "condition",
                 shapiro = TRUE,
                 title = x,
                 histogram = TRUE))

## [[1]]

## 
## [[2]]

## 
## [[3]]

## 
## [[4]]

## 
## [[5]]

## 
## [[6]]

## 
## [[7]]

## 
## [[8]]

## 
## [[9]]

## 
## [[10]]

## 
## [[11]]

Looks rather reasonable now, though not perfect (fortunately t-tests are quite robust against violations of normality). It seems though that for some reason our BSCS variable was made worse by the transformation. Let’s use the untransformed variable for this time then.

data$BSCS.BN <- scale(data$BSCS)

We will also standardize the variables based on the MAD.

# Standardize and center main continuous IV variable (based on MAD)
# data <- data %>%
#   mutate(across(all_of(col.list),
#                 ~scale_mad(.x),
#                 .names = "{col}"))

We can now resume with the next step: checking variance.

Homoscedasticity

# Plotting variance
plots(lapply(col.list, function(x) {
  nice_varplot(data, x, group = "condition")
  }),
  n_columns = 3)

Variance looks good. No group has four times the variance of any other group. We can now resume with checking outliers.

Outliers

# Using boxplots
plots(lapply(col.list, function(x) {
  ggplot(data, aes(condition, !!sym(x))) + 
  geom_boxplot()
  }),
  n_columns = 3)

There are some outliers, but nothing unreasonable. Let’s still check with the 3 median absolute deviations (MAD) method.

find_mad(data, col.list, criteria = 3)

## 5 outlier(s) based on 3 median absolute deviations for variable(s): 
##  blastintensity.BN,  blastduration.BN,  blastintensity.duration.BN,  blastintensity.first.BN,  blastduration.first.BN,  blastintensity.duration.first.BN,  KIMS.BN,  BSCS.BN,  BAQ.BN,  SOPT.BN,  IAT.BN,  
## 
## Outliers per variable:

## $SOPT.BN
##   Row   SOPT.BN
## 1  61 -3.802272
## 2  84 -2.870609
## 3  90 -2.870609
## 4 218 -3.802272
## 
## $IAT.BN
##   Row   IAT.BN
## 1  54 2.655326

There are 6 outliers after our transformations.

Winsorization

Visual assessment and the MAD method confirm we have some outlier values. We could ignore them but because they could have disproportionate influence on the models, one recommendation is to winsorize them by bringing the values at 3 SD. Instead of using the standard deviation around the mean, however, we use the absolute deviation around the median, as it is more robust to extreme observations. For a discussion, see:

Leys, C., Klein, O., Bernard, P., & Licata, L. (2013). Detecting outliers: Do not use standard deviation around the mean, use absolute deviation around the median. Journal of Experimental Social Psychology, 49(4), 764–766. https://doi.org/10.1016/j.jesp.2013.03.013

# Winsorize variables of interest with MAD
data <- data %>% 
  mutate(across(all_of(col.list), 
                winsorize_mad,
                .names = "{.col}.w"))

# Update col.list
col.list <- paste0(col.list, ".w")

Outliers are still present but were brought back within reasonable limits, where applicable. We are now ready to compare the group condition (Control vs. Mindfulness Priming) across our different variables with the t-tests.

t-tests

nice_t_test(data, 
            response = col.list, 
            group = "condition") %>% 
  nice_table(highlight = 0.10)

## Using Welch t-test (base R's default; cf. https://doi.org/10.5334/irsp.82). 
## For the Student t-test, use `var.equal = TRUE`. 
##  
## 

Interpretation: There is no clear group effect from our experimental condition on our different variables. However, there is a marginal effect of condition on blast intensity, whereas the mindfulness group has slightly higher blast intensity than the control group. Let’s visualize this effect.

Violin plots

Blast Intensity

nice_violin(data, 
            group = "condition", 
            response = "blastintensity.BN.w",
            comp1 = 1,
            comp2 = 2,
            obs = TRUE)

Blast Duration

nice_violin(data, 
            group = "condition", 
            response = "blastduration.BN.w",
            comp1 = 1,
            comp2 = 2,
            obs = TRUE)

Intensity * Duration

nice_violin(data, 
            group = "condition", 
            response = "blastintensity.duration.BN.w",
            comp1 = 1,
            comp2 = 2,
            obs = TRUE)

1st Intensity

nice_violin(data, 
            group = "condition", 
            response = "blastintensity.first.BN.w",
            comp1 = 1,
            comp2 = 2,
            obs = TRUE)

1st Duration

nice_violin(data, 
            group = "condition", 
            response = "blastduration.first.BN.w",
            comp1 = 1,
            comp2 = 2,
            obs = TRUE)

1st Intensity * Duration

nice_violin(data, 
            group = "condition", 
            response = "blastintensity.duration.first.BN.w",
            comp1 = 1,
            comp2 = 2,
            obs = TRUE)

Means, SD

Let’s extract the means and standard deviations for journal reporting.

Blast Intensity

data %>% 
    group_by(condition) %>% 
    summarize(M = mean(blastintensity),
              SD = sd(blastintensity),
              N = n()) %>% 
  nice_table(width = 0.40)

Blast Duration

data %>% 
    group_by(condition) %>% 
    summarize(M = mean(blastduration),
              SD = sd(blastduration),
              N = n()) %>% 
  nice_table(width = 0.40)

Intensity * Duration

data %>% 
    group_by(condition) %>% 
    summarize(M = mean(blastintensity.duration),
              SD = sd(blastintensity.duration),
              N = n()) %>% 
  nice_table(width = 0.40)

1st Intensity

data %>% 
    group_by(condition) %>% 
    summarize(M = mean(blastintensity.first),
              SD = sd(blastintensity.first),
              N = n()) %>% 
  nice_table(width = 0.40)

1st Duration

data %>% 
    group_by(condition) %>% 
    summarize(M = mean(blastduration.first),
              SD = sd(blastduration.first),
              N = n()) %>% 
  nice_table(width = 0.40)

1st Intensity * Duration

data %>% 
    group_by(condition) %>% 
    summarize(M = mean(blastintensity.duration.first),
              SD = sd(blastintensity.duration.first),
              N = n()) %>% 
  nice_table(width = 0.40)

Moderations

Let’s see if our variables don’t interact together with our experimental condition. But first, let’s test the models assumptions.

Assumptions

Blast Intensity

big.mod1 <- lm(blastintensity.BN.w ~ condition_dum*KIMS.BN.w + 
                 condition_dum*BSCS.BN.w + condition_dum*BAQ.BN.w + 
                 condition_dum*SOPT.BN.w + condition_dum*IAT.BN.w, 
               data = data, na.action="na.exclude")
check_model(big.mod1)

Blast Duration

big.mod2 <- lm(blastduration.BN.w ~ condition_dum*KIMS.BN.w + 
                 condition_dum*BSCS.BN.w + condition_dum*BAQ.BN.w + 
                 condition_dum*SOPT.BN.w + condition_dum*IAT.BN.w, 
               data = data, na.action="na.exclude")
check_model(big.mod2)

Intensity * Duration

big.mod3 <- lm(blastintensity.duration.BN.w ~ condition_dum*KIMS.BN.w + 
                 condition_dum*BSCS.BN.w + condition_dum*BAQ.BN.w + 
                 condition_dum*SOPT.BN.w + condition_dum*IAT.BN.w, 
               data = data, na.action="na.exclude")
check_model(big.mod3)

1st Intensity

big.mod4 <- lm(blastintensity.first.BN.w ~ condition_dum*KIMS.BN.w + 
                 condition_dum*BSCS.BN.w + condition_dum*BAQ.BN.w + 
                 condition_dum*SOPT.BN.w + condition_dum*IAT.BN.w, 
               data = data, na.action="na.exclude")
check_model(big.mod4)

1st Duration

big.mod5 <- lm(blastduration.first.BN.w ~ condition_dum*KIMS.BN.w + 
                 condition_dum*BSCS.BN.w + condition_dum*BAQ.BN.w + 
                 condition_dum*SOPT.BN.w + condition_dum*IAT.BN.w, 
               data = data, na.action="na.exclude")
check_model(big.mod5)

1st Intensity * Duration

big.mod6 <- lm(blastintensity.duration.first.BN.w ~ condition_dum*KIMS.BN.w + 
                 condition_dum*BSCS.BN.w + condition_dum*BAQ.BN.w + 
                 condition_dum*SOPT.BN.w + condition_dum*IAT.BN.w, 
               data = data, na.action="na.exclude")
check_model(big.mod6)

All the models assumptions look pretty good overall actually, even with all these variables. The lines for linearity and homoscedasticity are a bit skewed but nothing too crazy. Let’s now look at the results.

Moderations

Blast Intensity

big.mod1 %>% 
  nice_lm() %>% 
  nice_table(highlight = TRUE)

Blast Duration

big.mod2 %>% 
  nice_lm() %>% 
  nice_table(highlight = TRUE)

Intensity * Duration

big.mod3 %>% 
  nice_lm() %>% 
  nice_table(highlight = TRUE)

1st Intensity

big.mod4 %>% 
  nice_lm() %>% 
  nice_table(highlight = TRUE)

1st Duration

big.mod5 %>% 
  nice_lm() %>% 
  nice_table(highlight = TRUE)

1st Intensity * Duration

big.mod6 %>% 
  nice_lm() %>% 
  nice_table(highlight = TRUE)

Interpretation: The condition by trait self-control (brief self-control scale, BSCS) interaction comes up for all variables (so it must be somewhat reliable).

Interaction plots

Let’s plot the main significant interaction(s).

Blast Intensity

interact_plot(big.mod1, pred = "condition_dum", modx = "BSCS.BN.w", 
              modxvals = NULL, interval = TRUE, x.label = "Condition", 
              pred.labels = c("Control", "Mindfulness"),
              legend.main = "Trait Self-Control")

Blast Duration

interact_plot(big.mod2, pred = "condition_dum", modx = "BSCS.BN.w", 
              modxvals = NULL, interval = TRUE, x.label = "Condition", 
              pred.labels = c("Control", "Mindfulness"),
              legend.main = "Trait Self-Control")

Intensity * Duration

interact_plot(big.mod3, pred = "condition_dum", modx = "BSCS.BN.w", 
              modxvals = NULL, interval = TRUE, x.label = "Condition", 
              pred.labels = c("Control", "Mindfulness"),
              legend.main = "Trait Self-Control")

1st Intensity

interact_plot(big.mod4, pred = "condition_dum", modx = "BSCS.BN.w", 
              modxvals = NULL, interval = TRUE, x.label = "Condition", 
              pred.labels = c("Control", "Mindfulness"),
              legend.main = "Trait Self-Control")

1st Duration

interact_plot(big.mod5, pred = "condition_dum", modx = "BSCS.BN.w", 
              modxvals = NULL, interval = TRUE, x.label = "Condition", 
              pred.labels = c("Control", "Mindfulness"),
              legend.main = "Trait Self-Control")

1st Intensity * Duration

interact_plot(big.mod6, pred = "condition_dum", modx = "BSCS.BN.w", 
              modxvals = NULL, x.label = "Condition", interval = TRUE, 
              pred.labels = c("Control", "Mindfulness"), 
              legend.main = "Trait Self-Control")

Interpretation: The interaction is pretty much the same for all models. Counterintuitively, for people with low self-control, the priming mindfulness condition relates to lower aggression relative to the control condition. In contrast, for people with high self-control, the priming mindfulness condition relates to higher aggression.

Simple slopes

Let’s look at the simple slopes now (only for the significant interaction).

Blast Intensity

big.mod1 %>%
  nice_lm_slopes(predictor = "condition_dum",
                 moderator = "BSCS.BN.w") %>% 
  nice_table(highlight = TRUE)

Blast Duration

big.mod2 %>%
  nice_lm_slopes(predictor = "condition_dum",
                 moderator = "BSCS.BN.w") %>% 
  nice_table(highlight = TRUE)

Intensity * Duration

big.mod3 %>%
  nice_lm_slopes(predictor = "condition_dum",
                 moderator = "BSCS.BN.w") %>% 
  nice_table(highlight = TRUE)

1st Intensity

big.mod4 %>%
  nice_lm_slopes(predictor = "condition_dum",
                 moderator = "BSCS.BN.w") %>% 
  nice_table(highlight = TRUE)

1st Duration

big.mod5 %>%
  nice_lm_slopes(predictor = "condition_dum",
                 moderator = "BSCS.BN.w") %>% 
  nice_table(highlight = TRUE)

1st Intensity * Duration

big.mod6 %>%
  nice_lm_slopes(predictor = "condition_dum",
                 moderator = "BSCS.BN.w") %>% 
  nice_table(highlight = TRUE)

Interpretation: The effect of priming mindfulness on blast intensity is only significant for people with a high self-control.

Conclusions

Based on the results, it seems that the interaction of interest comes up for all six measures of blast aggression (intensity, duration, the combination of the two, and the first blast of each type), suggesting it is reliable.

The effect sizes are slightly lower for measures of first blast (sr2 = 0.3) than the average intensity or duration (sr2 = 0.4), or intensity * duration (sr2 = 0.5).

Therefore, based on the marginally larger effect size, perhaps it does make sense to use the intensity * duration combination in future studies. My intuition is also that the effect is more reliable for reactive aggression (all trials) than proactive aggression (first measure only). Another reason to avoid using only the first trials is that they lead to problems with the distributions (i.e., they are not normal and are difficult to transform to normality).

Preregistration

For the preregistration, I would suggest committing to using the average intensity * duration of all trials, as suggested above, and using the same design.

Package References

report::cite_packages(sessionInfo())

• 1. Peterson, R. A. (2021). Finding Optimal Normalizing Transformations via bestNormalize. The R Journal, 13:1, 310-329, DOI:10.32614/RJ-2021-041
• Andri Signorell et mult. al. (2021). DescTools: Tools for descriptive statistics. R package version 0.99.44.
• H. Wickham. ggplot2: Elegant Graphics for Data Analysis. Springer-Verlag New York, 2016.
• Long JA (2019). interactions: Comprehensive, User-Friendly Toolkit for Probing Interactions. R package version 1.1.0, https://cran.r-project.org/package=interactions.
• Lüdecke et al., (2021). performance: An R Package for Assessment, Comparison and Testing of Statistical Models. Journal of Open Source Software, 6(60), 3139. https://doi.org/10.21105/joss.03139
• Lüdecke et al., (2021). see: An R Package for Visualizing Statistical Models. Journal of Open Source Software, 6(64), 3393. https://doi.org/10.21105/joss.03393
• Makowski, D., Ben-Shachar, M.S., Patil, I. & Lüdecke, D. (2020). Automated Results Reporting as a Practical Tool to Improve Reproducibility and Methodological Best Practices Adoption. CRAN. Available from https://github.com/easystats/report. doi: .
• R Core Team (2022). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. URL https://www.R-project.org/.
• NA
• NA
• NA
• NA

References